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Key  Role  of  Application  Security  Risk 

Analysis  in  the  Cybersecurity  Game 

■  Ultimate  goal  is  to  prevent  security 
vulnerabilities  from  ever  entering  software 

■  Reality  is  they  are  already  there  and  even  new 
code  from  security-aware  developers  needs 
to  be  checked 

■  Application  security  risk  analysis  is  the 
practice  of: 

-checking  software  for  weaknesses/vulnerabilities 
-characterizing  the  risk  they  pose 
-identifying  and  prioritizing  mitigations 
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Varying  Perspectives  of  Analysis 


■  static  source  code 

■  static  binary  code 

■  dynamic  application  scanning 

■  application  penetration  testing 

■  application  data  security 

■  fuzzing 

■  complexity 

■  composition  &  pedigree 

■  etc. 
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Varying  Capabilities  of  Analysis 
Perspectives 

■  Different  perspectives  are  effective  at  finding 
different  types  of  weaknesses 
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Development  Institute 


■  Some  are  good  at  finding  the  cause  and  some  at 
finding  the  effect 
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Automating  Analysis  Perspectives 
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■  Automation  should  be  leveraged  wherever  possible 
but  should  be  combined  with  Tocused  manual 
analysis 


■  Automated  tools  will  find  the  low-hanging  fruit  much 
faster  than  manual  analysis  can 


■  Manual  analysis  will  find  less  obvious  and 
occasionally  high-risk  issues 
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Current  State  of  the  Practice 
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■  Most  organizations  undertaking  application 
security  risk  analysis  only  perform  one  or 
maybe  two  analysis  perspectives  and  those 
are  done  as  independent  processes  often  by 
separate  teams 

-  If  developer-centric  organization,  typically  start  with 
static  analysis 

-  If  test-centric,  typically  start  with  application  scanning 
and  penetration  testing 

-  If  information  assurance  or  data-centric,  typically  start 
with  data  security  scanning 
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The  Gestalt  of  Multi-perspective  Analysis 
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■  Better  situational  awareness 


■  Reinforce  confidence  in  findings  of  each 
perspective 


■  Combine  the  assurance  of  dynamic  analysis  with 
the  detail  of  structure  analysis  to  plan  effective 
mitigation  of  high-criticality  risk 
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The  Challenges  of  Integrated  Multi¬ 
perspective  Analysis 
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■  Varying  perspectives  have  different  drivers  and  priorities 
based  on  context 


■  Differing  perspectives  treat  “location”  of  issue  differently 
making  correlation  a  challenge 


■  Each  tool  for  each  perspective  has  its  own  reporting  schema 

■  Need  for  a  unified  findings  schema  (SAFES) 
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The  Need  for  Standards  in  Effective  Integration 
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■  Always  make  sure  comparing  apples  to  apples 

■  Weakness 

-  Common  Weakness  Enumeration  (CWE) 

■  Attack 

-  Common  Attack  Pattern  Enumeration  and  Classification 
(CAPEC) 

■  Vulnerability 

-  Common  Vulnerabilities  and  Exposures  (CVE) 

■  Technical  Context 

-  Common  Platform  Enumeration  (CPE) 

■  Mitigation 

-  Common  Control  Enumeration  (CCE) 
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A  Recommended  Baseline  for  Multi¬ 
perspective  Analysis 
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■  To  effectively  assess  the  security  risk  of  an 
application,  an  assessment  methodology  should  at 
a  minimum  include  the  following  perspectives: 

-  Static  source  code  analysis 

-  Application  scanning  &  penetration  testing 

-  Application  data  security  analysis 
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Static  Source  Code  Analysis 
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■  Analyze  code  without  executing  it 

■  Strengths 

-  Fast  compared  to  manual  code  review 

-  Fast  compared  to  testing 

Complete,  consistent  coverage  of  source  code  (all  paths) 
Brings  security  knowledge  with  it 

■  Limitations 

-  Only  analyzes  the  source  code  you  feed  it 

-  Doesn’t  find  everything 

■  Architecture  errors 

■  Bugs  you’re  not  looking  for 

■  System  administration  mistakes 

■  User  mistakes 
False  positives 

■  Multi-perspective  integration  value 

-  Actual  location  of  the  weakness  in  code 
Identify  issues  to  target  with  penetration  testing 

Identify  co-influencing  weaknesses  within  relevant  contexts 
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Application  Scanning  &  Penetration  Testing 
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■  Security  testing  (black  box)  of  applications  through  simulated 
attacks 

■  Strengths 

-  Simulates  the  actual  risk  (attacker’s  action) 

-  Tests  full  software  stack 
Low  false  positives 

-  Mature  technology 

■  Limitations 

Only  as  good  as  what  you  scan  (crawling  limitations) 

-  Analysis  limited  to  the  test  cases  executed 

-  Must  run  tests  often  to  stay  protected 

-  Can  only  be  performed  once  code  is  ‘runable’ 

Risky  to  run  on  production  applications 

Cannot  identify  the  actual  source  of  the  problem,  only  the  symptom 

■  Multi-perspective  integration  value 

-  Confirming  that  weaknesses  are  vulnerable 

-  Mapping  penetration  scans  to  locations  in  source  code 

Mapping  data  security  findings  to  injection  findings,  privilege  issues,  etc. 
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Application  Data  Security  Analysis 
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■  Analyzing  the  security  concerns  of  how  an  application  accesses 
and  manages  its  database 

■  Strengths 

Analyzes  a  live,  fully  configured  system  rather  than  just  source  code 
Good  at  catching  really  bonehead  mistakes  (they  are  more  common  than 
you  think) 

Helps  mitigate  both  insider  and  external  threats 

■  Limitations 

-  Only  as  good  as  what  you  tell  it  to  look  for 

-  Does  not  understand  semantics  of  data  (can  use  limited  proxies) 

■  Multi-perspective  integration  value 

-  Confirmation  of  likely  weaknesses  as  vulnerabilities 

-  Better  contextual  info  about  nature  and  severity  of  weaknesses 
Improved  understanding  of  likelihood  of  weaknesses  being  exploitable 

-  Increases  accuracy  of  forensic  data 
Improved  data  flow  policies 

-  Improved  Access  Control 
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Summary  and  Conclusions 
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■  Software  Assurance  analysis  is  increasingly  becoming  a 
high  priority  and  is  maturing  in  its  capability 

■  Varying  perspectives  of  analysis  are  available,  each  with 
their  own  unique  value 

■  Blending  multiple  perspectives  together  yields  better 
overall  coverage  and  an  integrated  gestalt 

■  It  is  real  and  possible  to  begin  pursuing  this  approach  today 
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Taming  the  Tower  of  Babel: 
Software  Assurance  Findings 
Expression  Schema  (SAFES) 


Sean  Barnum 
MITRE 
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Today's  Challenge 
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■  There  is  no  standard  reporting  format  for  SwA 
analysis 

-Very  difficult  to  combine  results  of  multi-perspective 
analysis 

-Very  difficult  to  combine  results  of  multi-tool  analysis 

-Very  inefficient  for  tool  vendors  looking  to  integrate  results 
with  other  tools  (very  costly  and  redundant) 

-Very  difficult  to  trend  across  assessments  from  different 
tools  or  analysts 

-Very  difficult  to  automate  meta-analysis  and  the 
assessment  process 
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SAFES  Effort 
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■  Software  Assurance  Findings  Expression  Schema  (SAFES) 


■  Sponsored  by  the  NSA  Center  for  Assured  Software  (CAS) 

■  Objectives: 

Enable  and  encourage  consistency  in  software  assurance  tool, 
service  and  analysis  practice  findings 

-  Establish  more  structured  and  effectively  useful  software 
assurance  tool,  service  and  analysis  practice  results 

Enable  integration  of  results  from  multiple  software  assurance 
tools,  services  or  analysis  practices 

Enable  automated  processing  of  software  assurance  tool,  service 
or  analysis  practice  results 
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SAFES  Approach 
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Phase  1 


Phase  2 


Future 


■  Community  collaboration 

■  Build  from  state  of  the  practice  (considered  ~20  tools  &  services) 

■  Enhance  with  state  of  the  art 

■  Define  a  comprehensive  schema  covering  all 
aspects  of  software  assurance  analysis  reporting 

■  Enable  &  demonstrate  practical  use 

■  Continually  refine  for  coverage,  consistency  & 
efficiency 

■  Layer  the  schema  into  a  framework  for 
composable  and  focused  use 

■  Strive  for  flexibility  and  extensibility 

■  Mature  towards  formalization 
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SAFES  Initial  Scope 

■  In-scope  perspectives  for  initial  effort: 

-  Static  source  code  analysis 

-  Static  binary  code  analysis 

-Web  application  penetration  testing 

-  Data  security  analysis 
—  Fuzzing 

—  Threat  modeling 
—  Architectural  risk  analysis 

■Some  vendors  actively  collaborating  others  were 
passively  incorporated 
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SAFES  is  a  comprehensive  and  detailed  schema 


■  Info  on  findings 

-  Description 
Categorization 

-  Location 

-  Prioritization 

-  Correlations 

■  Info  on  analysis  approach 

-  Tool  or  service 

-  Methodology 

-  Detection  mechanisms 


•  Info  on  mitigation 

•  Info  on  meta-analysis 

•  Info  on  personnel 

•  Info  on  application 

-  Structure,  content  & 
configuration 

-  Business/mission  and  security 
context 

•  Info  on  assurance  case 

•  Info  on  threat  analysis 
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Key  Constructs 


■  Sub-Assessment  scopes 

■  T  races 

■  Report  views 

■  Assurance  case 

■  Finding  prioritization 

■  Tool-Service  info 

■  Findings  correlations 
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A  Sampling  of  Potential  Use  Cases 
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Understand  the  Business  Context  of 
application 

Identify  risks 

Map  technical  risks  to  business  context 
Map  the  application  attack  surface 
Identify  relevant  threats 
Inventory  and  characterize  assets 
Create  threat  model 

Define  FISMA  security  categorization  (FIPS- 
199) 

FISMA  Security  Planning  (SP800-18) 

FISMA  Risk  Assessment  (SP800-30) 

Conduct  multi-tool/multi-perspective 
analysis 

Identify  false  positives 
Characterize  risk 
Prioritize  risk 


Correlate  findings 

Stitch  dynamic  &  static  location  results 
Integrate  automated  and  manual  analysis 
Reuse  common  mitigation  advice 
Create  assessment  report 
Create  different  versions  of  report 
Define  an  assurance  case  for  an  application 
Create  an  assurance  case  compliance  report 
Import  CWE  content  into  local  context 

Identify  common  finding  trends  across 
portfolio  by  technology  context 

Maintain  analysis  accountability 
Identify  trends  in  tool  and  rule  efficacy 

Mapping  between  various  tool  level 
definitions 
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SAFES  Maturation  Paths 
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■  Usability:  primarily  focused  on  efforts  surrounding  the 
schema  to  make  it  more  usable  by  the  community  such 
as  native  transforms,  tooling,  etc. 

■  Refinement:  primarily  focused  on  improving  the 
quality  and  coverage  of  the  schema  itself  with 
activities  such  as  adding  new  perspectives,  adding  new 
schemas,  fixing  errors,  etc. 

■  Formalization:  primarily  focused  on  gradually  (as 
quickly  as  is  prudent  and  accepted  by  the  targeted  user 
community)  incorporating  in  formal  standards-based 
approaches  (vocabulary,  structure,  etc.)  and  working 
towards  handoff  of  development  to  an  appropriate 
community  standards  consortium  body 
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SAFES  Phase  2 
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■  Develop  5-10  transforms  from  native  tool  output  to 
SAFES  (currently  for  CAS  internal  use  but  hopefully  will 
eventually  be  snared) 

■  Develop  a  demonstrative  use  case  example  for  SAFES 

■  Develop  lightweight  initial  prototype 
authoring/editing/reporting  tools  (very,  very  simple) 

■  Develop  a  real,  permanent  website  as  part  of  MSM 

■  Coordinate  with  standards  organizations  for  planning 
towards  future  maturation  and  formalization 
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SAFES  Next  Steps  Beyond  Phase  2 
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■  Identify  &  support  real-world  prototype  usage  of  SAFES 

■  Refine  based  on  feedback 

■  Refine  &  extend  authoring/editing/reporting  tools  with 
the  goal  of  eventually  transferring  this  work  to  other 
parties  (vendors,  open-source  projects,  consortia,  etc.) 

■  Incorporate  coverage  for  more  tools,  services  &  analysis 
practices 

■  Work  with  vendors  (and  OS  projects)  to  develop  more 
native  transforms  and  encourage  native  output  of  SAFES 

■  Refine  for  efficiency 

■  Refine  for  flexibility  (framework  layering) 

■  Refine  for  formalization  towards  existing  standards 
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Questions? 


Sean  Barnum 
MITRE 

sbarnum@mitre.org 
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